Methods and apparatus for 3D imaging and custom manufacturing

ABSTRACT

Apparatuses and systems for generating faithful 3D geometric models that correspond to the shape of an imaged 3D physical object; for storing, transmitting, and transforming those 3D models; for manufacturing 3D objects based upon those models; and for capture, transmission, storage, and transformation of the 3D models and manufacturing of objects from those models. Methods can include generating a 3D model by imaging a 3D object, transforming that 3D model with a transforming device to conform to a negative shape of the 3D object, using photogrammetry to create images from known non-repeatable positions, and using landmarks to identify common reference points in those images.

CROSS REFERENCE TO RELATED APPLICATION

The present application is a continuation application related to andclaims priority from prior non-provisional application Ser. No.14/104,303, received by the USPTO on Dec. 12, 2013, entitled “METHODSAND APPARATUS FOR 3D IMAGING AND CUSTOM MANUFACTURING”, the contents ofwhich are incorporated herein by this reference and are not admitted tobe prior art with respect to the present invention by the mention inthis cross-reference section. The specification of Ser. No. 14/104,303follows below, unchanged.

CROSS REFERENCE TO RELATED APPLICATION

The present application is related to and claims priority from priorprovisional application No. 61/739,729, received by the USPTO on 20 Dec.2012, entitled “METHODS AND APPARATUS FOR 3D IMAGING AND CUSTOMMANUFACTURING”, the contents of which are incorporated herein by thisreference and are not admitted to be prior art with respect to thepresent invention by the mention in this cross-reference section.

BACKGROUND OF THE INVENTION

This invention relates to providing methods and apparatus for improvedthree-dimensional (3D) imaging and custom manufacturing. Moreparticularly this invention relates to providing methods and apparatusmaking objects requiring very close fits for comfort, retention orperformance, wherein some body parts vary so significantly that even asmall number of standard sizes cannot address the requirement.

Before the industrial revolution, all items were hand crafted, thoughnot all were custom fit. During the industrial revolution, standardizedobjects became cheap to create due to automation. But custom fitsolutions remained expensive and made to order. For some objects, asingle size and style could meet the requirements of most buyers, andautomation was a good solution. But for items like shoes, one size fitsall was not a good compromise. Still the cost advantages of automationwere so substantial, that manufacturing in a few standard sizes was agood compromise to address both cost and fit. With the advent ofstandard sizing, commerce changed and you could purchase ready-madegoods such as shoes in a variety of sizes.

However, some objects require very close fits for comfort, retention orperformance, and some body parts vary so significantly that even a smallnumber of standard sizes cannot address the customer requirements. Oneobject that falls in this category is a custom fit earpiece such as istypically found on hearing aids. Failure to get a perfect seal causesundesirable acoustical results such as feedback. For this reason, customfit earpieces remain made-to-order. To date this has been an expensiveprocess largely performed by hand-craftsmanship.

For this reason low cost custom fit objects, manufactured on demand, andpotentially manufactured at the point of sale, has remained largely adream.

OBJECTS AND FEATURES OF THE INVENTION

A primary object of this invention is to provide a means of capturingand storing the three dimensional geometry objects.

A second object of this invention is to reduce the time, cost andexpertise required to capture, transmit and store such 3D geometries.

A third object of this invention is to use such captured 3D geometriesto manufacture and deliver to a consumer a custom object or custom fitobjects, on demand, at any desired location, including at the point ofsale.

There has long been a desire to create custom fit objects on demand, atthe point of purchase, but such automated manufacturing systemsgenerally do not currently exist. Thus, an additional object of thisinvention of this invention is fulfilling that desire by combining thenecessary technologies for capturing customized or personalized 3Dgeometries, and for manufacturing items designed to fit those geometriesexactly.

A further primary object and feature of the present invention is toprovide such a system that is efficient, inexpensive, and handy. Otherobjects and features of this invention will become apparent withreference to the following descriptions.

3. SUMMARY OF THE INVENTION

We here disclose methods and apparatus we have developed for capturingthe geometries of, and for manufacturing, three-dimensional objects.

The geometry of an object fitting within the imaging volume can becaptured three dimensionally (referred to herein as “scanning”). A widevariety of 3D objects can be scanned, e.g., a model, figurine, andbiometric features. Even biological and biometric features such asfingers, face, nose, or your ear can be imaged and their geometriescaptured. The 3D objects, once imaged are converted into 3D geometricmodels. The imaged objects may be replicated, or custom fit. Theirreplicas may be shaped to conform to the negative space (i.e., negativeshape) they create. For instance, captured ear geometries may be used todesign and manufacture custom fit earpieces. Once imaged, additionalmodifications may be done to the geometry model, eliminating the need tophysically modify a copied device or custom fit shape.

Object geometry data can be transferred across the Internet, or storedin the Internet servers (referred to as “in the cloud”), ready to bemanufactured at any desired location at any time. Manufacturing may bedone using additive manufacturing techniques (referred to as “3Dprinting”), or negative manufacturing techniques (e.g. “millingmachines”) in a variety of different materials.

By analogy to a fax machine that scans an image at one location andreproduces it at another, the invention allows objects to be 3D scannedat one location and reproduced at a different location. Or by analogy toa copy machine, it may reproduce a copy in the same location. Or byanalogy to a network of printers connected to a data storage device, itmay reproduce a copy at any later time at any of the 3D printer ormilling machine locations.

Objects may be manufactured at the same location as the scan takes placewith available materials and local 3D printer/milling machine stationcapabilities, enabling convenient replication near to the scanningpoint. Or they may be manufactured at a remote location where othermaterials or alternate fabrication methods may be available, and thenshipped to the consumer.

We also disclose a business process that enables a consumer to capture3D object geometries, to design and manufacture the 3D objects based onthose geometries. This can include not only strict replication of thescanned object, but also deriving new objects based on the scannedgeometries. A particular embodiment of this process supports makingcustom fit items based on scans of body parts, such as the manufactureof earpieces custom designed to fit the 3D shape of the purchaser'sears.

While most any object can be imaged and manufactured, in a preferredembodiment we image ears geometries that are used to manufacturecustom-fit earpieces. These earpieces fit the contours of your ears, andmay also be customized for hearing protection or for attachment to thespecific geometries of a variety of headsets for listening to a mediaplayer, phones, tactical communications, hearing aids and other audiodevices or sources.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features and advantages of the invention will be morereadily understood from the following detailed description of theinvention that is provided in connection with the accompanying drawings,in which:

FIG. 1 shows a flow chart of the Custom Fit manufacturing Process.

FIG. 2 shows a system diagram and data flow chart, showing the unitsthat comprise the invention.

FIG. 3 shows a human ear and the 4 arrows point at 4 areas where shadowsare likely if the ear is scanned from only a single vantage point.

FIG. 4 shows a human ear and the 4 circles are at the 4 points where lowwattage lights may be placed next to the skin to diffuse through theskin and illuminate transdermally the structures that lie underneath theridge of skin.

FIG. 5 shows four different mechanisms for positioning cameras forimaging a 3D object from multiple locations.

FIG. 6 shows a sample imaging target ring surrounding an ear.

FIG. 7, it shows a sample right panel background.

FIG. 8 shows a sample left panel background.

FIG. 9 shows a sample front panel background.

FIG. 10 shows a sample back panel background.

FIG. 11 shows a sample floor.

DETAILED DESCRIPTION OF THE INVENTION

Referring to FIG. 1, it shows a flow chart of a custom fit manufacturingprocess as follows.

-   -   1. Step 100: A user desiring to have a custom object        manufactured for him or her arrives at the custom fit        manufacturing station.    -   2. Step 110: The user positions an object, e.g., a body part, to        be imaged in the imaging zone (an imaged object).    -   3. Step 120: The imaging zone acquires two-dimensional (2D)        images of the object and any other necessary information needed        to digitally reconstruct the object's 3D geometry. The necessary        information acquired depends on imaging method used, but may        include absolute or relative positions of the cameras and light        sources and their expected imaging pattern when imaging a flat        plane.    -   4. Step 130: A computational unit converts the 2D images and        necessary information into a 3D imaged object model of the        imaged object.    -   5. Step 140: The computational unit stores the 3D imaged object        model in the storage unit. This 3D imaged object model data may        be used immediately (via step 170) to make one or more of the        custom objects, or it can be stored for use at a later time        (steps 150 and 160).    -   6. Step 150: If no more of the custom objects will be        manufactured at this time, the process is suspended until the        user wishes to manufacture the custom objects based on the        stored 3D imaged object model.    -   7. Step 160: Once the user's 3D imaged object model is in        storage it may be retrieved at any future time, to manufacture a        new custom object, e.g., a custom-fit earpiece or a casting mold        for the desired object. When the user desires to manufacture the        new custom object, the 3D imaged object model data are passed to        the next available manufacturing unit at the time of        manufacture.    -   8. Step 170: Upon request by the user when he or she is ready to        manufacture the new custom object, the 3D imaged object model is        retrieved into a computation unit where it is transformed into        the 3D object or 3D custom-fit object model. The parameters of        this custom fit model may be set by the user or by a technician,        or determined automatically, depending on application and user        preferences.    -   9. Step 180: The resulting 3D custom-fit object model, e.g., a        custom-fit earpiece data model, is transferred from the        computation unit to a 3D Manufacturing station for        manufacturing. The 3D custom-fit object model data may be stored        as a permanent data storage file if the user is likely to create        another just like it later, or it may be transferred as a        temporary data store if it is likely to only have a one-time        use.    -   10. Step 190: The 3D manufacturing station, which may be a 3D        additive manufacturing device (e.g., 3D printer), or another        computer-aided design (CAD)/manufacturing system such as a        Computer Numerical Control (CNC) machine, then manufactures a        custom or custom-fit 3D object according to the specification in        its 3D object model. When manufacturing is complete, the 3D        manufacturing station returns to a suspense state (step 150)        until the next manufacturing request is made.

Referring to FIG. 2, it shows a system diagram and data flow chart,showing the elements that comprise the invention. Note that althoughonly one unit of each type is shown, the invention includesconfigurations with multiple imaging units, computational units, datastorage units and manufacturing units. These units may be physicallyintegrated in a single physical package, or they may be distributed andconnected by a communications network, such as WiFi in a store, or viathe Internet, enabling the computational unit to use cloud computationby remote servers and the data storage unit to use cloud data storageservices.

-   -   200: Imaging Unit. Imaging unit, 200, is composed of one or more        imaging devices, such as the cameras or scanners capable of        generating 2D or 3D image files of an object, depending on the        3D technology applied. The cameras and light sources may move in        order to sense the geometry of the object from different planes        cut through the object, or to capture from different viewpoints,        or fixed cameras may be used. Various light sources may be        placed at one or more viewpoints or a combination of fixed and        moving elements of the imaging unit may be used. The imaging        unit may store the generated image files in the data storage        unit for later access, or pass them directly to the        computational unit.    -   210: Image Files. Image file, 210, generated by the imaging unit        is used by the computational unit to create 3D models of the        imaged objects that have not already been rendered.    -   220: Computational Unit. Computational unit, 220, may be a        personal computer (PC), a server, or specialized hardware for        computing and transforming 2D images into a 3D model. The        computational unit interacts with the data storage unit, the        imaging unit, and the 3D manufacturing unit.    -   230: Imaged Object 3D Model. Imaged object 3D model, 230,        generated by the computational unit, holds personalized 3D        geometry information defining shape of the imaged object, e.g.,        jewelry, a figurine, or a human ear.    -   240: Data Storage Unit. Data storage unit, 240, holds 3D models        computed from the imaging data. These models may then be        transformed into custom fit 3D data models that may or may not        be stored long term. The data storage units may also store the        imaging data prior to use by a computational unit.    -   250: Custom Object 3D Model. Custom object 3D model, 250,        produced by the computational unit is derived from the geometry        of the imaged object. The custom object 3D model is transformed        as necessary for a desired purpose, e.g., the final custom shape        or custom-fit of the imaged object, e.g., jewelry, models or        figurines, or a custom-fit earpiece designed for a specific user        and headset.    -   Step 260: 3D Manufacturing Unit. 3D manufacturing unit, 260, may        be the 3D additive manufacturing station (3D printer), the CNC        machine or other device that can manufacture the 3D object        according to instructions contained in the 3D model.

Referring to FIG. 3, showing a human ear (ear) and the 4 arrows pointingat 4 areas where shadows are likely if the ear is scanned from only asingle vantage point. The circles show horizontal locations where thebest vantage point for positioning imaging and illumination sources toavoid shadows along the line of the attached arrow. A preferredembodiment includes multiple vantage points for imaging and illuminationfor ensuring sufficient imaging of the imaged object, i.e., the ear, andespecially to capture sufficient biometric data of the ear to create acustom fit earpiece.

-   -   300: Location 300 is a good location for a vantage point and        illumination to see into inter-tragus notch at base of bowl of        the ear.    -   310: Location 310 is a good location for a vantage point and        illumination to see into helix notch, where it overlaps        anti-helix of the ear.    -   320: Location 320 is a good location for a vantage point and        illumination of a visible part of ear canal.    -   330: Location 330 is a good location for a vantage point of        ridge of the

-   anti-helix that sometimes extends over concha bowl.

Referring to FIG. 4, it shows a human ear and the 4 circles are at the 4points where low wattage lights may be placed next to the skin todiffuse through the skin and illuminate transdermally the structuresthat lie underneath the ridge of skin.

-   -   400: Location 400 is a good location for illuminating the        inter-tragus notch at the base of the bowl of the ear, and the        lower back of the concha under the anti-helix.    -   410: Location 410 is a good location for illumination of the        helix notch, where it overlaps the anti-helix.    -   420: Location 420 is a good location for illumination of the        entrance to the ear canal.    -   430: Location 430 is a good location for illuminating under the        ridge of the anti-helix that sometimes extends over the upper        back of the concha bowl.        Continuing to refer to FIG. 4, it shows locations 400, 410, 420,        and 430 where lights may be placed for transdermal illumination        of the surrounding areas of the ear.

Referring to FIG. 5, it shows 4 types of mechanisms for positioning thecameras for imaging the 3D object from multiple locations.

-   -   500: Robot arm, 500, enables you to swing around the object        being imaged to see it from any vantage point, enabling you to        find the best viewpoint to see into concave areas of the 3D        object.    -   510: A series of cameras, 510, placed on a hemispherical arm        enable imaging the object from fixed elevations. By rotating the        arm around the object it can be imaged from any azimuth        direction.    -   520: By adding a second arm, 520, that pivots around a 45 degree        elevation, a single camera can view the 3D object from any        elevation between 0 degrees (equator) and 90 degrees (zenith).        If the arm is placed at the equator, any elevation between 90        degrees (zenith) and −90 degrees (nadir) can be imaged. By        rotating the arm around the object you can achieve any azimuth        as well. Thus through a combination of the hemispherical arm and        the second arm enable achieving full or nearly full        hemispherical coverage. If the imaged object is on a pedestal or        stake, the arms may be pivoted below the equator and thus        captured with nearly full spherical coverage, allowing capture        of the top and bottom of the object in a single model.        Alternatively a object may be imaged in one orientation and then        flipped vertically to capture the other side, and the two        resulting models can be stitched together by software during the        modeling step.    -   530: For applications where movement of the 3D object between        frames is a concern, the 3D object can be imaged simultaneously        from multiple fixed vantage point cameras.

Referring to FIG. 6, it shows a sample imaging target ring surroundingan ear. The imaging target ring has a number of marks on it which act aslandmarks that allow the photogrammetry software to synchronize the samelandmarks as seen from different vantage points. The imaging target ringalso has markings of known size that allow the 3D models to be properlyscaled.

Referring to FIG. 7, it shows a sample right panel background.Backgrounds are landmark target rich for the photogrammetry software toidentify many landmark points so that we can triangulate welleverywhere. This is especially important when the 3D object itself issmooth and lacks few target points. Because the imaged 3D object isbetween the camera and the floor or walls, we are better able todetermine the precise edges from each vantage point. In our preferredembodiment we image in a hemispherical pattern, focused on the center ofthe floor of our box. At high elevation angles the camera will see onlythe floor and 3D object, but at low elevation angles this wall will beseen behind the 3D object. In our preferred embodiment we use highcontrast nature photographs rather than synthetic or constructed images,because the photogrammetry landmark recognition models have beenoptimized for naturally occurring organic shapes. Each wall is acompletely different image from each other wall to avoid any landmarkconfusion during modeling that could occur if there was repetition orsymmetry.

Referring to FIG. 8, it shows a sample left panel background. As withthe right panel, front panel and back panel, this panel will be imagedfrom some camera positions when the camera is at a low elevation angle.This image shares characteristics of the other wall and floor designs.It is high contrast, irregular (random), non-repeating, andnonself-similar at different scales. Images that repeat or areself-similar at different scales can make it easy for the photogrammetrysoftware to confuse two distinct different points as the same landmark,which will result in a distorted model of the 3D object. Selection ofimages such as these avoid these problems.

Referring to FIG. 9, it shows a sample front panel background. As withthe right panel, left panel and back panel, this panel will be imagedfrom some camera positions when the camera is at a low elevation angle.This image shares characteristics of the other wall and floor designs,but is of a different object to ensure its landmarks won't erroneouslybe mapped to the same point on other walls. Differences in size or shapeof the wall images are determined by the precise dimensions of thescanner, which may vary based on the 3D object being imaged.

Referring to FIG. 10, it shows a sample back panel background. As withthe right panel, left panel and front panel, this panel will be imagedfrom some camera positions when the camera is at a low elevation angle.This image shares characteristics of the other wall and floor designs,but is of a different 3D object to ensure its landmarks won'terroneously be mapped to the same point on other walls. Differences insize or shape of the wall images are determined by the precisedimensions of the scanner, which may vary based on object being imaged.

Referring to FIG. 11, it shows a sample floor background. As with thewall panel backgrounds, the floor background provides a target richenvironment for finding landmarks that can be matched from image toimage. Because the 3D object is placed on the floor parts of the floorare always visible unless the camera elevation is on the equator. Thethin lines on the floor (circle and star pattern) are used in testing toverify cameras are operating from desired positions, and to initiallyset the dimensional scale and axes of the scanner using an image objectof known dimensions and position. By specifying the exact dimensions andpositions of known landmark points on the floor, we are able reverse thetriangulation process to determine the precise camera positions whereimages were taken with respect to the same reference axes and scale. Westore these positions in a “template” that we can use to provideautomated scaling and cropping during normal processing. Our mechanicalsand electronics ensure we use the same camera positions each time, thuswe now know the precise camera positions used each time. Given the knowndistances between cameras, and the angles to common reference points indifferent images we are now able to accurately determine the scale ofthe unknown 3D object. Also since we know the actual distance of thefloor and walls from the cameras, we can automatically crop the floorand walls from the model leaving only the intended 3D object modeled.

In a preferred embodiment, we can image either physical impressions ofthe ear (a convex object), or the ear itself (a concave object).

Creating a good 3D model of the ear is especially difficult for severalreasons. Human skin is translucent and intense incident light, such asfrom a laser tends to both diffuse and reflect. This poses a problem forscanners relying on laser or structured light technologies that useincident light.

For this reason, in a preferred embodiment the inventors prefer usingphotogrammetry for capturing ear geometries, which has not been used forthis purpose before.

However, a problem for photogrammetry is that for good results, you needto identify point landmarks from multiple vantage points that you canuse in triangulation. Except for exceptionally freckled humans, mostears are largely featureless, which would make it difficult forphotogrammetry to yield accurate results, which is one reasonphotogrammetry has not been used before for capturing ear geometries.Moreover, physical impressions taken from the ears are generallymonochromatic, smooth with continuously varying curves, making it hardto find landmarks on the imaged target object. However, the inventorswere able to work around this limitation by surrounding the ear with aprinted ring or bowl marked with landmark points that enablephotogrammetry to accurately identify these points, and when imagingimpressions we captured images within an enclosed space with specialbackgrounds that provide many of the landmarks used for triangulationeven though the target object lacks them. By imaging the ear and thesurrounding landmarks we are able to capture the visible shapes withlandmark points.

Additionally photogrammetry has not been used for capturing ears becausephotogrammetry requires the image to contain an object of known scale toget the size right. Since ears don't naturally have any features ofprecisely known size, this is a problem for photogrammetry. However,because we created the landmark points on a surrounding ring or bowlhaving a known size, we solved the scaling problem. This use of thelandmark points enables photogrammetry as an acceptable solution forcapturing ear geometries.

Additionally, because the photogrammetry apparatus consistently takesimages from the same positions, we capture a test object of a knownfixed size. And, because we already know the scale of the 3D object,rather than using triangulation to find its size and distance, wereverse the triangulation to find the distance of each camera from someof the registration points. This allows us to calculate the cameralocations with precision. Since the device does not change its operatinglocations, we are able to detect the camera positions relative to eachother.

As mentioned before, a disadvantage of laser line and structured lightscanning methods for ear scanning is the translucency of skin whichdiffuses the incident light so it is no longer a well-defined point. Anadvantage of this embodiment using photogrammetry is that translucencyof skin can actually be employed to better capture parts of the ear thatare hard to directly illuminate.

We have identified four major ear features that must be well captured toachieve a good fit for custom fit earpieces. They are: (1) the outerpart of the ear canal from the concha to the second bend, (2) the helixlock where the helix goes over the anti-helix, (3) the intertragal notchbetween the tragus and anti-tragus, and (4) the concha under theanti-helix ridge.

Again, referring to FIG. 3, which identifies camera and illuminationpositions for best imaging these areas, the size of the lightingelements and the cameras may make it difficult to simultaneouslyilluminate and image from these positions. And in some cases the ridgesso completely curl over the concha or entrance to the ear canal thatillumination of the area is nearly impossible. This is mostsignificantly a problem around the ear canal.

An advantage of this preferred embodiment for an ear scanner is that byusing photogrammetry we can turn the translucency of the skin from adisadvantage into an advantage. We can place light sources, such as lowwattage LEDs proximate to the skin in these areas. The light willdiffuse through the obscuring ridge of skin and illuminate the interiortransdermally. For instance a light source proximate to the tragal foldwill diffuse through that skin and illuminate the outer part of the earcanal. Proper placement of light sources can assist with transdermallyilluminating the intertragal notch (with sources over the tragal foldand intertragus), the concha (on the antitragus and antihelix, and ofthe helix lock with illumination placed on top of the helix). Becausethese sources are outside of the concha area, they do not obscureimaging the interior of the concha from any viewpoint. They can in factbe hidden by the aforementioned ring or bowl that provides scale andlandmark synchronization information for photogrammetry.

A preferred embodiment relies on multiple imaging vantage points. Thereare multiple ways to image from these vantage points, and differentchoices may be preferred based on whether the application is imaging anear (subject to motion) or an ear impression.

Use of multiple cameras simultaneously capturing the target, can beadvantageous for capturing ears, since it minimizes the time that usermust interact with the scanner, and also minimizes the possibility ofthe ear moving between images which would confound the photogrammetricmodeling.

For an object such as an ear impression, or piece of jewelry that isstationary, motion between image captures is not an issue, and cost maybe reduced by reducing the number of the cameras and moving them betweenmultiple viewpoints. At the limit a single camera may be employed,although depending on time vs. cost vs. redundancy tradeoffs othernumbers of the cameras may be used in combination with motion.

There are many ways the cameras may be driven to their desiredlocations. These include a robot arm that can trace any path, rotationalarms that drive the cameras around the object in an azimuth direction,elevational arms that can move the cameras up and down in elevation,combinations of azimuthal and elevational arms, or even hemispherical orhelical screw tracks.

A preferred embodiment puts together the components necessary to makelow cost custom fit manufacturing, including manufacturing at point ofsale, a reality.

We image an object fitting inside the scanning zone. These may be simpleobjects like models, figurines or jewelry or complex objects likebiometric features such as ears. Captures of 3D ear geometries, eitherdirectly from ears, or indirectly from negative space impression moldstaken from ears, are used to design and manufacture custom fit earpiecesusing Computer Aided Design (CAD) and Computer Aided Manufacturing (CAM)software, and manufacturing technologies such as Additive Manufacturing(3D Printers) or computer numerically controlled machines that usenegative manufacturing techniques.

Our invention can apply these methods to creation of many custom fitobjects, but the manufacturing time can vary with the complexity, volumeand number of materials comprising the final object. An object as smalland simple as an earpiece or piece of jewelry could be manufactured inunder an hour with existing manufacturing technologies, while an objectas large or as complex as high performance shoe would likely take muchlonger with existing manufacturing techniques. For this reason, thispreferred embodiment of the invention initially creates small, simple,single material goods such as jewelry, models, figurines, and custom fitearpieces. As manufacturing technology improves, we expect to producemore complex multiple material goods that have other components embeddedin the product to enhance its usefulness or value, but stillmanufactured in a timely manner.

A preferred embodiment of this invention begins by addressing thequestion “Why is it so hard to get custom fit earpieces?”. Over the lasteighty years the hearing aid industry evolved from a standard “one sizefits all” earpiece, to custom fit earpieces. One of the primary factorsthat led to the move to custom fit earpieces was the miniaturization ofthe hearing aid devices, which increasingly put the microphones closerand closer to the speakers.

This is a concern for the designer of hearing aids, because hearing aidsmust amplify the signal a great deal. But if the amplified sound reachesthe microphone, “feedback” occurs and a painful screeching soundresults. If a secure seal of the ear canal can be achieved, theamplified sound directed to the ear drum can be acoustically isolatedfrom the microphone on the other side and the possibility of feedback isreduced. For this reason, the needs of the hearing aid industry has beendriving the development of methods and apparatus for the capture of eargeometries and for manufacturing custom fit earpieces.

But once the technology existed for capturing ear geometries and producecustom fit objects, applications to other areas such as industrialhearing protection, music and communications also became possible.

Because of this connection to the growing hearing aid industry, thetraditional way to produce such custom ear plugs, earpieces, adaptersand components for hearing aids or communications devices has requiredconsumers wanting such devices to visit a trained audiologist, who wouldinject a fast hardening putty into each of the consumer's ears. Afterthe putty hardens (typically in 15-30 minutes), the audiologist wouldship the ear impressions to a custom manufacturer. The manufacturerwould then use the consumer's ear molds to construct an inverse mold,and then use the inverse mold to make a new object that was identical inshape to the molds that came from the consumer's ears. In the case of ahearing aid, music listening device or communications device, additionalmanufacturing steps might be required to construct a hole that coulddirect sound from the device's speaker through the custom earpiece to apart of the ear canal proximate to the ear drum.

The newly manufactured earpiece would be sent back to the originalaudiologist who would ask the patient to schedule another appointmentwhere the audiologist would validate that the earpieces weresufficiently snug to prevent the feedback problem, yet not so tight thatthey were painful to wear. The high demands of these conflictingrequirements, and the multi week process between each mold taking andfitting appointment made obtaining a very good impression the first timevery important. This took some skill, especially to navigate the syringearound two separate “bends” in each ear canal, and without damaging thedelicate eardrum. Capturing the deep geometry of the ear canal, allowingthe hearing aid designer to place the end of the device as close to theear drum as possible, and as far away from the microphone as possiblewas critical to success of hearing aid fittings.

One disadvantage of this method is that the original putty molds shrinkover time, thus if the consumer needs a replacement earpiece or is fitfor a different device at a future time, the consumer must replicatetheir visit to the audiologist and a new mold will have to be made.

With the advent of laser 3D geometry scanners, some manufacturersdecided that rather than make the 2nd inverse mold they would use alaser scanner and create a digital 3D geometry data model of eachoriginal ear impression mold. Since the geometry model does not changeover time the way the original molds do, this can be used to generateadditional copies on demand without new molds. However, as the ear is apart of the human body which continues to grow over a lifetime, frequentre-captures are typically necessary in the Hearing Aid business where atight fit to avoid feedback is most critical. A preferred embodiment ofthis invention facilitates capture of the geometries of such physicalmolds so they do not need to be shipped.

In addition, this growth rate is slow enough that in less demandingapplications such as music listening or communications where feedback isnot an issue, earpieces made based on geometries digitally captured at asingle fitting could be adequate for many years or even decades.

Even with growing applications in noise protection and communicationsthat were less demanding in their demands for capturing deep eargeometries, because of the potential damage to the ear drum if moldimpression taking technique is poor, getting the initial ear moldscontinued to done primarily by trained medical professionals causing theinitial mold taking process to be an expensive, inconvenient and timeconsuming process.

A preferred embodiment comprises a new way to capture this informationthat does not involve using putty, but would instead use optical meansto capture ear geometries. This was further developed and perfected intothe invention, which is disclosed below.

Methods for capturing 3D geometry information and building 3D CAD modelsare not new. Among the common methods for capturing 3D geometries aremoving laser line scanners, structured light scanners, andphotogrammetry to name a few. All of these methods for capturing 3Dpoint cloud information can be used with preferred embodiments of thisinvention.

However, a problem for any imaging system is the problem of shadows. Ifthe surface cannot be seen from the vantage point of the imaging device,or is not illuminated by an illumination device, the 3D model will havea defect. This is why 3D scanning is typically only used for capturingconvex shapes which can be viewed from a single vantage point, or wherethe object can be placed on a carousel that rotates in a predictable wayin front of a fixed camera and light source.

In terms of manufacturing an object that should fill the negative spaceof a concavity (i.e., negative shape), the defects (areas not imaged ornot imaged well) in the negative shape will result in voids between themanufactured object and the imaged object.

To ensure that we do not suffer from significant optical shadowing, wecapture images from multiple vantage points to maximize accurate 3Drendering.

Photogrammetry is the science of determining 3D geometric propertiesfrom 2D photographic images. Photogrammetry does an excellent job ofrecreating 3D objects. In its simplest embodiment, stereo cameras imagean object from two locations. Geometrical positions illuminated andvisible to both the cameras can be determined by running rays from eachcamera to the common point. By adding more vantage points andcircumnavigating the object we can determine the geometry of the entireobject. In fact, entire cityscapes can be captured with images takenwhile circling or in multiple flyovers.

The same technology, photogrammetry, is used to create 3D films for themovie industry. Photogrammetry techniques are very good for capturingshapes; however, unless distances between vantage points are knownprecisely, they depend on knowing the size of features in the images toscale properly. One embodiment of the invention that uses photogrammetrytakes advantage of the apparatus to ensure that we know the scale of theimaged objects by first deriving and then reusing known camera positionsto yield both scale and better precision in the shape capture.

Laser scanning technology has been around for decades and is anotherexcellence source capable of 3D rendering. It depends on having a laserand camera at a known distance apart. A point on the surface of theobject illuminated by the laser reflects to the camera. We know by thelaws of optics that the angle of incidence equals the angle ofreflectance. Therefore if we know the angle of the laser relative to thebase line connecting the laser and camera, we know two angles and oneside of the triangle they form. By triangulation, we can calculate thedistances and thereby derive the geometry. By creating a laser line, wecan capture many points with a single image and thus speed up theprocess.

A newer image technique called Structured-Lighting extends thisprinciple even further, painting many lines across the surface inpatterns that allow the system to resolve which visible line correspondsto which transmitted light and angle, thus allowing multiple “lines” tobe captured in a single image.

A disadvantage of the laser and structured light methods is that theyrely on painting the surface of the object with bright light. If theilluminated surface reflects or diffuses light, a single incident lightray may wind up illuminating multiple reflected points, or the area ofdiffusion, making it hard to determine the precise location of theilluminated point with accuracy. This is particularly a concern forimaging human skin, such as the surface of the ear, since it is subjectto both diffusion of bright light and irregular reflections.

These and many more methods exist for acquiring 3D geometries.

While the combination of the 3D scanner and 3D printer or CNC machine isuseful for duplicating a shape or its inverse, the 3D printer can printany 3D model, including ones that are designed from scratch without aninitial scan. By placing a 3D printer or CNC machine at a point of salelocation, on demand manufacturing with mass customization reaches a newlevel of convenience, and eliminates shipping costs and delays.

A preferred embodiment of this invention consists of an imaging unit, acomputation unit, a data storage unit and a 3D manufacturing unit. Theimaging system, computation unit, and printer may all be tightlyintegrated in a single device, or they may be fielded as multipledevices that intercommunicate. For instance, some computations may bedone by servers on the Internet, and data may also be stored on remoteservers, and 3D scanners may be at one location but 3D printers used inmanufacturing may be in yet another location.

A user of the invention places an object to be imaged at the focal zoneof the imaging unit. For instance, in one embodiment jewelry, models,household décor, figurines or physical ear impressions are placed in theimage zone and a 3D model of the imaged object is created in order tocreate an exact replica. In a preferred embodiment, an ear is imaged inorder to create an object that is not a replica of the ear, but ratherto a custom fit earpiece that fills the negative space of the concha(bowl) of the ear.

The output of this imaging unit is a set of 2D images. The image datamay be stored in a data storage unit before being passed on to thecomputation unit.

The 2D images are converted into 3D imaged object models using commonalgorithmic image analysis tools that calculate 3D geometries usingphotogrammetry, laser line triangulation, structured light or other 3Dimaging methods. This 3D data transformation is done by a computationunit. The resulting 3D imaged object model is stored in the data storageunit.

The 3D imaged object model (in a preferred embodiment: a 3D ear model)is then translated by the computation unit, using 3D CAD software, tosmooth edges, fill areas not well captured, remove excess material, toextrude extra material to provide a place for anchoring any attachments,or perform other customization desired. This translation creates thedesired 3D Custom object model that will be used in manufacturing. In apreferred embodiment where we create custom fit earpieces, thesetransformations are typically to create an anchor hole to hold the postof a headset, and to route a sound tunnel from the end of the headsetthrough the earpiece into the ear canal. The output of thistransformation step is a 3D Custom object model.

In the final step of the fabrication process of a preferred embodimentof this invention, we convert the 3D custom object model into a finishedmanufactured object using automated 3D manufacturing methods. Thesefabrication processes take place in the 3D manufacturing unit. In apreferred embodiment, we create finished products such as: jewelry,models, household décor, figurines, or earpiece are made using 3Dadditive manufacturing and printing of the desired object. However, inother embodiments of the invention can use subtractive methods such asCNC machines and other manufacturing technologies to develop products tomeet customer needs.

OPERATION

The User positions the object to be fitted (e.g. jewelry, models, household décor, figurines, physical ear impression or an ear), into theimaging zone of the apparatus. The imaging apparatus captures images ofthe object.

In the next step, the captured 2D images are output from the imagingunit into the computational unit. This process converts the captured 2Dimages into a 3D geometry model of the imaged object to be fit orreplicated. We use a pattern background that avoids replication orself-similar elements to provide a target rich environment fortriangulating even if the object being imaged lacks its own landmarks.We also can add additional landmarks to the target object by sprinklingpowder, painting a surface pattern using pigment or light, or using adifferent material containing contrast markers embedded within the moldmaking material to ensure that the molds so produced have lots ofoptical landmarks.

In the following step, the 3D imaged object model is stored in the datastorage device for retrieval when the user wishes to manufacture acustom-fit device. In most cases this data is immediately used as inputto the object transformation process, although once stored the processmay be repeated at this step to create additional custom or custom-fitobjects as replacements for use with new accessories (e.g. differentheadset models) without the need for re-imaging.

In the succeeding step, the 3D model of the imaged object is transformedinto a 3D model of the desired custom object to be manufactured. Thismay be an entirely automated computation process, or may involveadditional input from a technician depending on the complexity of thetransformation desired.

Once the desired 3D custom or custom-fit object model has been produced,it is passed to the 3D manufacturing apparatus that then physicallyreplicates the custom fit object based on its 3D model. This finalmanufacturing step may be a simple single step: a direct to 3D printoperation; or it may involve multiple steps including making of castingmolds and subsequent casting of parts, or various finishing operations.

ALTERNATIVES

There are many kinds of cavities and surfaces that can be 3D scannedusing preferred embodiments of this invention, and many custom fitobjects can be manufactured from the resulting 3D geometry files, as wedisclose here.

In a preferred embodiment of the invention the application of thesemethods and apparatus are used to scan the 3D geometries of human ears,and to create custom fit ear pieces that may be used for noise reductionand hearing protection; or as part of a personal or portable sounddelivery system such as a music player in-ear headset; or as part of anear piece for a communications system, such as a bluetooth cellularphone, call center telephone or tactical radio walkie-talkie.

While there are several advantages to applying the invention to thisspecific application, there is nothing about the invention which wouldlimit its application to 3D imaging of other objects, nor tomanufacturing physical replicas or custom-fit or custom designed objectsof other types.

For instance, a preferred embodiment of this invention can be applied toseveral other medical applications, including capture of the 3D geometryof other body parts, such as feet, vaginal, nasal, oral or analcavities, or combined with laparoscopy, of internal body cavities orblood vessels. From these 3D scans, custom fit orthotics, vaginal, oralor anal inserts, stents and other custom fit 3D objects can be produced.

Beyond medical applications, these same techniques can be used toproduce custom fit personalized jewelry, garments, or sculptures andother adornments, plus home décor, figurines or sculptures.

Furthermore, preferred embodiments of this invention are not limited tothe scanning of biological surfaces or production of custom fit objectsonly for use in biological contexts. Preferred embodiments of thisinvention can also be applied to creating a custom fit patch to a holeor crack in a ceiling, floor, wall or other surface, or to join two ormore objects securely.

While our preferred implementation of this invention uses opticalimaging methods, it should be understood that our invention can workusing non-visible electromagnetic radiation as well, including infrared,ultraviolet, microwave and radio waves and x-rays. It may also beextended to other forms of energy waves such as ultra-sound or computedtomography where 2D images are transformed into 3D images.

We claim:
 1. A method of making a second 3-dimensional object that fitsto a first 3-dimensional object to be fitted comprising: a) a method ofcreating a 3-dimensional geometry model comprising imaging said first3-dimensional object to be fitted using an imaging device; and b) amethod of transforming said 3-dimensional geometry model into saidsecond 3-dimensional object with a transforming device; c) wherein saidtransforming device is structured and arranged to select said second3-dimensional object conforming to negative shape of first said3-dimensional object or conforming to positive shape of said first3-dimensional object; d) wherein the method of creating said3-dimensional geometry model comprising imaging said first 3-dimensionalobject is provided with said imaging device comprising: i)photogrammetry apparatus configured for creating more than one imagefrom more than one known non-repeatable position; and ii) landmarksstructured and arranged for identifying more than one common referencepoint in multiple images using any combination of target coloring,background, pattern, and registration point markers relating to saidfirst 3-dimensional object; iii) wherein said common reference pointsexist in said multiple images are identified with respect to first andsecond background images occluded by said first 3-dimensional object atdistinct angles, wherein one or more of said common reference points arenot found in both said first and second background images; iv) whereinscale of said first 3-dimensional object is calculated from saidregistration point markers at said known non-repeatable 3-dimensionalpositions; v) whereby enabling using triangulation to find distance ofeach camera from selected registration points associated with said3-dimensional object to be fitted.
 2. The method of claim 1, wherein themethod of creating said 3-dimensional geometry model comprising imagingsaid first 3-dimensional object is provided with an illuminationapparatus comprising: i) selecting an indirect light source that isstructured and arranged with selected multiple wavelengths to enhancetransdermal diffusion of its light; ii) wherein said indirect lightsource is placed proximate to an area to be imaged of said first3-dimensional object; iii) wherein said indirect light source isstructured and arranged to transmit said selected multiple wavelengthsof light for indirectly illuminating said area to be imaged throughtransdermal diffusion of light from said indirect light source; iv)whereby transmitted said selected multiple wavelengths of light diffusesinto said area to be imaged.
 3. The method in claim 1, wherein themethod of creating said 3-dimensional geometry model comprising imagingsaid first 3-dimensional object further comprises a method for capturingsaid 3-dimensional geometry model of said first 3-dimensional object tobe fitted comprising: i) covering a first surface of said first3-dimensional object to be fitted with optically contrasting landmarks;ii) wherein said optically contrasting landmarks comprise applying lightand shadow in a repeating pattern; iii) wherein said opticallycontrasting landmarks are structured and arranged for identifying morethan one common reference point in said multiple images; iv) whereinsaid common reference points exist in each of said multiple images; v)whereby enabling registration points associated with said first surfaceof said first 3-dimensional object.
 4. The method in claim 1, whereinthe method of creating said 3-dimensional geometry model comprisingimaging said first 3-dimensional object further comprises a method forcapturing said 3-dimensional geometry model of said first 3-dimensionalobject to be fitted comprising: i) covering a second surface of saidfirst 3-dimensional object to be fitted with said op-tically contrastinglandmark; ii) wherein said optically contrasting landmark comprisescontrasting particles struc-tured and arranged to create random landmarkpoints embedded within said optically contrasting landmarks; iii)whereby said random landmark points are introduced in otherwisefeatureless smooth regions; iv) wherein embedded said random landmarkpoints are embedded at depths that are optically-thin relative to saidselected multiple wavelengths' optical penetration into said3-dimensional object to be fitted with said optically contrastinglandmark; v) wherein said optically contrasting landmarks are structuredand arranged for identifying more than one common reference point insaid multiple images; vi) wherein not all identified said commonreference points exist in each of said multiple images; vii) wherebyenabling registration points associated with said second surface of said3-dimensional object to be fitted.
 5. The method in claim 1, wherein themethod of creating said 3-dimensional geometry model comprising imagingsaid first 3-dimensional object further comprises a method for capturingsaid 3-dimensional geometry model of said first 3-dimensional object tobe fitted: a) wherein said first 3-dimensional object to be fitted isimaged against a patterned back-ground of non-repeating andnon-self-similar elements that is not masked out prior to building apoint cloud; b) whereby said patterned background of non-repeating andnon-self-similar elements are structured and arranged to provide atarget rich environment for selecting said landmarks that can beaccurately ranged; and c) wherein the method of creating said3-dimensional geometry model comprising imaging said first 3-dimensionalobject further comprises a method for improving capture of said imagesof said first 3-dimensional object to be fitted by imaging said3-dimensional object: i) wherein said first 3-dimensional object to befitted is imaged using said selected multiple wavelengths of lightexhibiting varying bright spots and shadows; ii) wherein said opticallycontrasting landmarks are structured and arranged for identifying morethan one common reference point in said multiple images; iii) whereinnot all identified said common reference points exist in each of saidmultiple images; iv) whereby said multiple wavelengths of lightexhibiting varying bright spots and shadows are structured and arrangedto provide a target rich environment for selecting said landmarks thatcan be accurately ranged; d) whereby said second 3-dimensional objectcustom fits with said first 3-dimensional object to be fitted.
 6. Themethod of claim 1 wherein the method of creating said 3-dimensionalgeometry model comprises a shape capture: a) wherein multiple fixedposition cameras are structured and arranged for capturing multipleimages simultaneously; and b) wherein scale of said 3-dimensional objectto be fitted is determined by first using known fixed camera positionsbuilt into the imaging device to yield both scale and precision in saidshape capture.
 7. The method of claim 6 wherein an imaging apparatussurrounds said 3-dimensional object to be fitted and provides images ofknown size that can be used to scale said 3-dimensional object to befitted.
 8. The method of claim 7 wherein said imaging apparatussurrounds said 3-dimensional object to be fitted with an calculatingapparatus that is structured and arranged for: a) providing said commonreference points; b) imaging both said 3-dimensional object to be fittedand said common reference points for creating said 3-dimensionalgeometry model; c) calculating all 3-dimensional shapes for creatingsaid 3-dimensional geometry model; and d) digitally removing said commonreference points and any surrounding objects not inclusive of said3-dimensional object to be fitted; e) whereby creating said3-dimensional geometry model.
 9. The method of claim 8 wherein themethod of creating said 3-dimensional geometry model comprises imaging:a) wherein said photogrammetry apparatus is configured for creatingmultiple image view-points that circumnavigate a path around said3-dimensional object to be fit-ted, using photogrammetry; b) whereinmultiple fixed position cameras are structured and arranged forcapturing multiple images simultaneously comprising said shape capture;and c) wherein scale of said 3-dimensional object to be fitted isdetermined by first deriving and then reusing known camera positions toyield both scale and precision in said shape capture.
 10. The method ofclaim 8 wherein the method of creating said 3-dimensional geometry modelcomprising imaging said 3-dimensional object to be fitted with an saidimaging device comprises digitally storing and retrieving said3-dimensional geometry model in a form required for replicating thatpart that enables using additive manufacturing technologies.
 11. Themethod of claim 8 wherein the method of creating said 3-dimensionalgeometry model comprising imaging said 3-dimensional object to be fittedwith an said imaging device and using additive manufacturing processesto build said custom objects based on stored said 3-dimensional geometrymodel.
 12. The method of claim 11 wherein said custom objects comprisecustom fit earpieces, ear plugs, ear tips, sleeves or shells thatsurround an audio or hearing protection product manufactured usingadditive manufacturing methods.
 13. The method of claim 1 wherein themethod of creating said 3-dimensional geometry model comprising imagingsaid 3-dimensional object to be fitted utilizing an analytical apparatuscomprising: a) an imaging unit structured and arranged for acquiringmore than one 2-dimensional image file; b) a computational unitstructured and arranged for transforming said 2-dimensional image fileinto said 3-dimensional geometry model of said 3-dimensional object tobe fitted; c) a data storage unit structured and arranged for storingsaid 3-dimensional geometry model; d) a computational unit structuredand arranged to translate said 3-dimensional geometry model of said3-dimensional object to be fitted into a 3-dimensional model of anintended custom object; and e) at least one 3-dimensional additivemanufacturing unit structured and arranged to produce a 3-dimensionalphysical object based upon said 3-dimensional model of said intendedcustom object.
 14. The method of claim 13 wherein any unit of saidanalytical apparatus may be physically separated from any other unit andin which there exists a communication link interconnecting all saidunits of said apparatus.
 15. The method of claim 1 wherein the method oftransforming the 3-dimensional geometry model into said 3-dimensionalobject comprises a method for manufacturing said custom objectcomprising: a) capturing 2-dimensional images of a 3-dimensional objectto be fitted; b) producing said 3-dimensional geometry model of theimaged object from said 2-dimensional images; c) transforming said3-dimensional geometry model of said 3-dimensional object to be fittedinto a 3-dimensional geometry model of said custom object to bemanufactured; and d) manufacturing said custom object based on said3-dimensional geometry model of said custom object to be manufacturedusing said 3-dimensional additive manufacturing process.
 16. The methodof claim 1 wherein said 3-dimensional object to be fitted that lackssuitable reference points for generating said 3-dimensional model viaphotogrammetry comprising: a) surrounding said 3-dimensional object tobe fitted with a reference point apparatus that provides such referencepoints; b) imaging both said 3-dimensional object and the surroundingreference points together in the same images; c) calculating3-dimensional shapes of the combined imaged area; and d) and digitallyremoving the surrounding object from said 3-dimensional model; e)whereby generating said 3-dimensional model of said 3-dimensional objectto be fitted, but lacking suitable reference points for generating said3-dimensional model via photogrammetry.
 17. The method of claim 1wherein said 3-dimensional object to be fitted lacks images of knownscale and further comprises: a) using photogrammetry by surrounding said3-dimensional object to be fitted with a scale apparatus that providessuch objects or images of known scale; b) imaging both said3-dimensional object to be fitted and the surrounding reference pointstogether in the same images, calculating the scale of the combinedimaged area; and c) digitally removing the surrounding object from themodel.
 18. The method of claim 1 comprising a method for determiningscale of said 3-dimensional object to be fitted comprising: a) imagingan object of known dimensions; b) using reverse light ray tracing todetermine location of said light sources and said camera locations thatare structured and arranged for ensuring scaling of the captured object;c) using known said camera positions to determine scale of said3-dimensional object to be fitted and location of any surrounding floorand wall images; and d) cropping said surrounding floor and wall imagesfrom said model, leaving only said 3-dimensional geometry model; e)whereby determining said scale of said 3-dimensional object to befitted.
 19. The method of claim 1, wherein the first and secondbackground images include high contrast nature photographs.
 20. Themethod of claim 1, wherein the first and second background imagesinclude images collected from a hemispherical pattern of directions,said hemispherical pattern of directions being focused on a position ofa floor or wall.